Report code churn across the codebase by typeless · Pull Request #196 · typeless/putup

typeless · 2026-07-28T00:01:48Z

A per-PR rework number answers "is this change rewriting recent work" — a property of the change. The question worth a standing metric is a property of the tree: how much of what we wrote recently is already gone?

The metric

Over a window ending at the analyzed commit: of the lines written in that window, how many are no longer present.

One git log --numstat --since pass gives what was written per file; one blame pass over those files counts what survived; the difference is work written and then discarded or rewritten inside the same window. Dated against the analyzed commit rather than wall-clock now, so a re-run agrees.

On a pull request it analyzes the PR head, so the reading is the state of the tree if this lands — not a score for the PR.

### Code churn (whole codebase, last 30d)

| Files | Lines written | Still present | Churned | Churn rate |
|---|---|---|---|---|
| 145 | 9916 | 8695 | 1221 | 12.3% |

Where the churn is

File	Lines written then discarded
`src/cli/cmd_build.cpp`	249
`.github/scripts/compose_metrics.py`	143
`.github/scripts/bench_metrics.sh`	92
`src/core/layout.cpp`	69
`src/core/new_delete.cpp`	63
…

Top of the list is src/cli/cmd_build.cpp at 249 lines written then discarded — where the identity and join work has been concentrated. The metric finds the area we already knew had been rewritten repeatedly, which is the check that it measures something real.

Stability

window	written	churned	rate
7d	3230	471	14.6%
14d	9335	1151	12.3%
30d	9916	1221	12.3%
90d	10399	1296	12.5%

Stable across windows, and highest over the shortest one — what you would expect if fresh code is what gets revised. The number is not an artifact of the window choice.

Exclusions, and how one of them was found

Tests, third_party/, examples/, and the generated bootstrap scripts.

Tests are rewritten whenever the code they cover is, so counting them measures the same production change twice. test/ is 408 of 694 tracked files.
third_party/ is vendored — its history is imports, not our work.
examples/ is build recipes for other people's source.
bootstrap-linux.sh / bootstrap-macos.sh earned their exclusion by ranking 2nd and 3rd on the first run. make bootstrap regenerates them from the Tupfiles, so their churn is a mechanical echo — at roughly ten times the volume — of a change already counted at its source.

That last one is the argument for looking at the output before trusting it: it was 242 and 241 lines of pure noise sitting directly under the top result.

Failure behaviour

Every git call degrades to empty output rather than raising, so a missing history would report zero churn instead of failing — silence and "nothing churned" look identical. A shallow clone, an unknown revision, and a window containing no production changes each refuse with a message. Verified against a real --depth 1 clone.

Cost

2.1s, inside the existing pr-metrics-comment job — which already checks out and only runs on pull requests, so no new job and no artifact round-trip. It needs fetch-depth: 0, since neither blame nor a 30-day log works in a shallow clone.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

Added + deleted lines measures how big a PR is, which the diff already says. The question worth asking in review is different: is this change writing new code, or rewriting code we wrote recently? A pattern of the second is how an encoding problem shows up -- the same area fixed again and again, each fix reasonable on its own. The metric is rework: of the lines this PR deletes, how many were themselves written inside the window (default 30 days). Ages come from blaming the merge base, and are measured against the merge base's own commit time rather than wall-clock now, so re-running CI later reports the same number. Reported alongside it: the median age of deleted lines, which is the sharper signal of the two -- near zero means the change is undoing work from the same few days -- and the files in the diff that other commits in the window also deleted from. Hotspots count commits that DELETED from a file, not commits that touched it. By the touch definition test/unit/test_e2e.cpp led every commit measured (38-46) for the uninteresting reason that every PR appends a scenario to it; by the delete definition it drops to 7 and the files actually being rewritten come first. The rework percentage is withheld below 20 deleted lines. A two-line diff that happens to touch recent code is not "100% rework", and a ratio over a handful of lines says nothing. Calibration against this repo's own history, which is what the metric is for: Make stale removal file-granular (fixing that day's work) 85.7% 0.0d Split command identity into key and signature 67.6% 11.2d Delete StringId's relational operators (21 old sites) 37.5% 121.7d No baseline artifact: comparing a rework ratio against the single previous commit on main is noise, and the median age is self-explanatory without calibration. A trailing-window comparison would be the way to add one. Runs in the existing pr-metrics-comment job, which already checks out and only runs on pull requests, so churn needs no artifact round-trip -- only fetch-depth 0, since blame cannot see history in a shallow clone. Costs 0.6s on the largest commit in recent history. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

Every git call in cmd_churn degrades to empty output rather than raising, which is right for the one case where emptiness is the answer (blaming a file the PR adds), but it means a shallow clone or an unreachable base would be reported as zero churn instead of as a failure. Silence and "nothing was reworked" looked identical. Both now refuse: a shallow repository names the fetch-depth it needs, and a base with no merge base against head says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

github-actions · 2026-07-28T00:16:46Z

PR metrics

Performance (gcc example, Linux)

Workload	Instructions	CPU time	Page faults	D1 miss	LL miss	Wall	Peak RSS
parse	1076 M	0.51 s	14.2 k	0.8%	0.1%	0.524 s	31.2 MB
dry-run	1881 M	0.62 s	16.6 k	0.9%	0.1%	0.635 s	40 MB (-0.2MB)

Deterministic signals: instructions (cachegrind-simulated instruction reads — exact across runs, no PMU needed), page faults, peak RSS, and the cachegrind D1/LL miss rates. CPU time is user+sys from time(1).

Internal statistics (gcc example, up-to-date dry run)

Metric	Value
Tupfiles parsed	24
Commands	3545
Commands scheduled	0
Files checked	5818
Files changed	0
Files in index	6087
Graph edges	367913
Index size (bytes)	7378282
Implicit deps	330624
Hash computations	0
Hashes skipped (stat cache)	5818
Stat calls	5869
Parse time (ms)	503.6 (+0.4%)
Total time (ms)	613.8 (+0.7%)
Runner CPU	AMD EPYC 7763 64-Core Processor

Counters from putup -n --stat on the fully-built gcc example (up-to-date dry run): deterministic work measures — a jump in commands scheduled, hash computations, or stat calls is a real behavior change, not noise. Timings are the minimum over repeated runs, compared only against a baseline from the same CPU model; the counters are the regression signal.

Binary size (Linux)

Binary	.text	.data	.bss	File
`putup`	517.6 KB	1.6 KB	98.7 KB	615.2 KB

Code churn (whole codebase, last 30d)

Files	Lines written	Still present	Churned	Churn rate
119	7492	6728	764	10.2%

Of the lines written across the codebase in the last 30 days, how many are already gone — work that was written and then discarded or rewritten inside the same window. This is the state of the tree including this PR, not a measure of the PR itself. Only code we write is counted: tests, examples, vendored and generated files, CI plumbing and prose are excluded. 3562 lines were deleted in the window in total, most of them older than it.

Where the churn is

File	Lines written then discarded
`src/cli/cmd_build.cpp`	247
`src/core/layout.cpp`	69
`src/core/new_delete.cpp`	63
`src/cli/context.cpp`	55
`src/graph/builder.cpp`	35
`src/core/string_pool.cpp`	34
`src/core/heap_buf.cpp`	33
`src/platform/vm-posix.cpp`	22
`src/cli/options.cpp`	21
`include/pup/core/layout.hpp`	15

Test coverage (lines)

Overall	Median file	Min file	Max file
86.7%	95%	0.0% `src/platform/path-posix.cpp`	100.0% `include/pup/core/arena.hpp`

103 files · 15163/17494 lines covered

Deltas vs main@fc14f89de.

Updated for 6d0e89b

Tests are rewritten whenever the code they cover is, so counting them measured the same production change twice and diluted the signal with churn that is working as intended. test/ is 408 of the repo's 694 tracked files, which is why it dominated every reading. Excluding it moves the numbers without changing the ranking: before after Make stale removal file-granular 112 rm 85.7% 109 rm 85.3% Split identity into key and signature 142 rm 67.6% 104 rm 69.2% Delete StringId's relational operators 24 rm 37.5% 24 rm 37.5% The last row is unchanged because that PR removed nothing from tests at all. The hotspot list goes to ten, which no longer fits a sentence, so it renders as a table inside a <details> block rather than pushing the summary out of view. Ties now break by path so the list is stable between runs rather than depending on which file the diff happened to list first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

A purely additive PR removes nothing, so there are no line ages to take a median of -- and the table printed "0.0 d". That is the strongest churn reading the metric can produce, shown for the situation with the least churn possible. The same cell said the same thing whether a change was undoing that morning's work or adding a new file. An empty measurement now reports n/a in the TSV and renders as an em dash, which keeps it distinct from the measured zero that matters: fixing code written the same day still reads 0.0 d, and that is the reading the metric exists to surface. The related failure is worse and is now an error rather than a number. If a diff removes lines but none of them can be dated -- blame failing on a rename, a base that is present but unreadable -- the ages list is empty for a reason that has nothing to do with churn, and the old code reported 0 reworked lines out of a real removal count. It now exits nonzero naming how many lines it could not date. Counts were correct throughout and are unchanged: this PR is 199 added, 0 removed against main, which git agrees with, because its later commits only edited lines the PR itself introduced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

…gate hotspots The table and the file list answer different questions over different ranges, and nothing said so. The counts are the PR's own diff, merge-base..head. The file list is the same files' history reachable from the merge base -- so it is what happened BEFORE the PR, deliberately excluding the PR's own commits, and it is context for the diff rather than part of it. Both now say which they are. The hotspot rule counts only commits that deleted from a file. That was introduced because test/unit/test_e2e.cpp led every reading purely by being appended to, and tests are now excluded outright, so the rule was worth re-justifying rather than keeping out of habit. It still discriminates: over a recent window, 59 of 102 production files count differently under touched-vs-deleted-from, and four are append-only with more than one touch -- include/pup/core/string_id.hpp, include/pup/cli/context.hpp, include/pup/core/region.hpp and src/index/writer.cpp. string_id.hpp is the clearest: two commits added deleted operators to it and neither removed anything, so it grows without being rewritten. Under the touched rule it would report as a hotspot. The measurement is recorded at the site. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

A per-PR rework number answers "is this change rewriting recent work", which is a property of the change. The question worth a standing metric is a property of the tree: how much of what we wrote recently is already gone. Churn is now, over a window ending at the analyzed commit: of the lines written in that window, how many are no longer present. One `git log --numstat --since` pass gives what was written per file; one blame pass over those files counts what survived; the difference is work written and then discarded or rewritten inside the same window. Dated against the analyzed commit rather than wall-clock now, so a re-run agrees. On a pull request it analyzes the PR head, so the reading is the state of the tree if this lands -- not a score for the PR. window written churned rate 7d 3230 471 14.6% 14d 9335 1151 12.3% 30d 9916 1221 12.3% 90d 10399 1296 12.5% Stable across windows, and higher over the shortest one, which is what you would expect if fresh code is what gets revised. Exclusions are now tests, third_party/, examples/ and the generated bootstrap scripts. The bootstrap scripts earned their exclusion by ranking 2nd and 3rd on the first run: `make bootstrap` regenerates them from the Tupfiles, so their churn is a mechanical echo, at roughly ten times the volume, of a change already counted at its source. Top of the list is src/cli/cmd_build.cpp at 249 lines written then discarded, which is where the identity and join work has been concentrated -- the metric finds the area we already know has been rewritten repeatedly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

.github/ is how the project is built and checked, not the thing being built, and its churn was inflating the rate by about three points: 13.1% with it, 9.8% without. Five of the top ten entries were CI scripts, three of them files that no longer exist because they were consolidated into this one -- real work, rewritten, but not where a build system's defects come from. Excluding it left TESTING.md ranked tenth, which is the same category error one step further out: prose is not code. Markdown accounts for 483 of the lines written in the window. Excluded by suffix, since docs live both under docs/ and at the root, along with docs/ itself. The top ten is now entirely src/, led by cmd_build.cpp at 249 lines written then discarded -- the identity and join work -- followed by layout.cpp, new_delete.cpp and context.cpp. window files written churned rate 7d 49 2525 398 15.8% 14d 115 7335 756 10.3% 30d 119 7502 770 10.3% 90d 119 7886 844 10.7% Still stable across windows and still highest over the shortest, so the narrower denominator has not made the number fragile. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLob4Ef3qyMd3Dnp9q3XmA

typeless and others added 2 commits July 28, 2026 08:00

typeless and others added 4 commits July 28, 2026 08:47

typeless changed the title ~~Report code churn as rework, not volume~~ Report code churn across the codebase Jul 28, 2026

typeless merged commit 618a627 into main Jul 28, 2026
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Report code churn across the codebase - #196

Report code churn across the codebase#196
typeless merged 7 commits into
mainfrom
worktree-churn-metric

typeless commented Jul 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

typeless commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The metric

Stability

Exclusions, and how one of them was found

Failure behaviour

Cost

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR metrics

Performance (gcc example, Linux)

Internal statistics (gcc example, up-to-date dry run)

Binary size (Linux)

Code churn (whole codebase, last 30d)

Test coverage (lines)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

typeless commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading